<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="generator" content="rustdoc">
    <title>Wayland and LVGL on PinePhone with Ubuntu Touch</title>

    
    <!-- Begin scripts/articles/*-header.html: Article Header for Custom Markdown files processed by rustdoc, like chip8.md -->
<meta property="og:title" 
    content="Wayland and LVGL on PinePhone with Ubuntu Touch" 
    data-rh="true">
<meta property="og:description" 
    content="Learn about Wayland and Ubuntu Touch on PinePhone... And how we build PinePhone Apps with LVGL" 
    data-rh="true">
<meta property="og:image" 
    content="https://lupyuen.github.io/images/wayland-title-small.jpg">
<meta property="og:type" 
    content="article" data-rh="true">
<!-- End scripts/articles/*-header.html -->
<!-- Begin scripts/rustdoc-header.html: Header for Custom Markdown files processed by rustdoc, like chip8.md -->
<link rel="alternate" type="application/rss+xml" title="RSS Feed for lupyuen" href="/rss.xml" />
<link rel="stylesheet" type="text/css" href="../normalize.css">
<link rel="stylesheet" type="text/css" href="../rustdoc.css" id="mainThemeStyle">
<link rel="stylesheet" type="text/css" href="../dark.css">
<link rel="stylesheet" type="text/css" href="../light.css" id="themeStyle">
<link rel="stylesheet" type="text/css" href="../prism.css">
<script src="../storage.js"></script><noscript>
<link rel="stylesheet" href="../noscript.css"></noscript>
<link rel="shortcut icon" href="../favicon.ico">
<style type="text/css">
    #crate-search {
        background-image: url("../down-arrow.svg");
    }
    a {
        color: #77d;
    }
</style>
<!-- End scripts/rustdoc-header.html -->


</head>
<body class="rustdoc">
    <!--[if lte IE 8]>
    <div class="warning">
        This old browser is unsupported and will most likely display funky
        things.
    </div>
    <![endif]-->

        <!-- Begin scripts/rustdoc-before.html: Pre-HTML for Custom Markdown files processed by rustdoc, like chip8.md -->

    <!-- Begin Theme Picker -->
    <div class="theme-picker" style="left: 0"><button id="theme-picker" aria-label="Pick another theme!"><img src="../brush.svg"
        width="18" alt="Pick another theme!"></button>
        <div id="theme-choices"></div>
    </div>
    <script src="../theme.js"></script>
    <script src="../prism.js"></script>
    <!-- Theme Picker -->

    <!-- End scripts/rustdoc-before.html -->
    

    <h1 class="title">Wayland and LVGL on PinePhone with Ubuntu Touch</h1>
    <nav id="TOC"><ul>
<li><a href="#how-x11-works">1 How X11 works</a><ul></ul></li>
<li><a href="#wayland-on-ubuntu-touch">2 Wayland on Ubuntu Touch</a><ul></ul></li>
<li><a href="#render-opengl-graphics-with-wayland">3 Render OpenGL Graphics with Wayland</a><ul></ul></li>
<li><a href="#get-egl-context-from-wayland">4 Get EGL Context from Wayland</a><ul></ul></li>
<li><a href="#build-and-test-wayland-app-on-linux">5 Build and Test Wayland App on Linux</a><ul></ul></li>
<li><a href="#fetch-wayland-interfaces">6 Fetch Wayland Interfaces</a><ul></ul></li>
<li><a href="#render-opengl-bitmap-texture-with-wayland">7 Render OpenGL Bitmap Texture with Wayland</a><ul></ul></li>
<li><a href="#lvgl-toolkit-for-graphical-user-interfaces">8 LVGL Toolkit for Graphical User Interfaces</a><ul></ul></li>
<li><a href="#port-lvgl-to-wayland">9 Port LVGL to Wayland</a><ul></ul></li>
<li><a href="#build-lvgl-on-pinephone-with-ubuntu-touch">10 Build LVGL on PinePhone with Ubuntu Touch</a><ul></ul></li>
<li><a href="#inject-lvgl-into-file-manager-app">11 Inject LVGL into File Manager App</a><ul></ul></li>
<li><a href="#run-lvgl-on-pinephone-with-ubuntu-touch">12 Run LVGL on PinePhone with Ubuntu Touch</a><ul></ul></li>
<li><a href="#overcome-apparmor-security-on-ubuntu-touch">13 Overcome AppArmor Security on Ubuntu Touch</a><ul></ul></li>
<li><a href="#what-i-like-about-ubuntu-touch-on-pinephone">14 What I like about Ubuntu Touch on PinePhone</a><ul>
<li><a href="#ubports-on-ubuntu-touch-wayland-and-mir">14.1 UBports on Ubuntu Touch, Wayland and Mir</a><ul></ul></li>
<li><a href="#gnome-and-gtk-on-wayland">14.2 GNOME and GTK on Wayland</a><ul></ul></li>
<li><a href="#wayland-on-xfce">14.3 Wayland on Xfce</a><ul></ul></li></ul></li>
<li><a href="#touch-input-for-lvgl-on-wayland">15 Touch Input for LVGL on Wayland</a><ul></ul></li>
<li><a href="#whats-next">16 What's Next?</a><ul></ul></li>
<li><a href="#configure-ssh-on-pinephone">17 Configure SSH on PinePhone</a><ul>
<li><a href="#generate-ssh-keys">17.1 Generate SSH Keys</a><ul></ul></li>
<li><a href="#install-ssh-keys">17.2 Install SSH Keys</a><ul></ul></li>
<li><a href="#start-ssh-service">17.3 Start SSH Service</a><ul></ul></li></ul></li>
<li><a href="#copy-files-from-microsd-card-on-pinephone">18 Copy Files from MicroSD Card on PinePhone</a><ul></ul></li>
<li><a href="#build-and-test-lvgl-app-on-linux">19 Build and Test LVGL App on Linux</a><ul></ul></li></ul></nav><p><img src="https://lupyuen.github.io/images/wayland-title.jpg" alt="Work-in-progress LVGL GUI Framework ported to Wayland EGL on PinePhone with Ubuntu Touch" /></p>
<p><em>Work-in-progress LVGL GUI Framework ported to Wayland EGL on PinePhone with Ubuntu Touch</em></p>
<p><strong>We ❤️   Old Underwear...</strong></p>
<p>They feel comfy, they fit our contours. Nevermind the holes and the old stains 🤢</p>
<p><strong>X11 is like Old Underwear.</strong> It's been around for 30 years... Yet we still use it in spite of its feature gaps and wonky quirks.</p>
<p><a href="https://wiki.pine64.org/PinePhone"><strong>PinePhone on Ubuntu Touch feels like... New Underwear.</strong></a></p>
<p>It runs Linux but it has none of the legacy X11 code. Because it's optimised for a great mobile experience with <strong>Wayland.</strong></p>
<p>But New Underwear feels uncomfortable. So today we'll learn Wayland and understand how apps are built with Wayland.</p>
<p>Hopefully someday we'll move on to newer, simpler app frameworks (like <a href="https://lvgl.io/">LVGL</a> and Flutter) as we discard our Old Underwear: X11, SDL, GTK, Qt, ...</p>
<p>The source code for this article may be found here...</p>
<ul>
<li>
<p><a href="https://github.com/lupyuen/lvgl-wayland"><code>github.com/lupyuen/lvgl-wayland</code></a></p>
</li>
<li>
<p><a href="https://github.com/lupyuen/pinephone-mir"><code>github.com/lupyuen/pinephone-mir</code></a></p>
</li>
</ul>
<h1 id="how-x11-works" class="section-header"><a href="#how-x11-works">1 How X11 works</a></h1>
<p><a href="https://en.wikipedia.org/wiki/X_Window_System">X11</a> is the Graphical Display Service that runs on most Linux desktops and notebooks.</p>
<p>Let's hunt for the X11 Service on Pinebook Pro...</p>
<p><img src="https://lupyuen.github.io/images/wayland-pinebook.png" alt="X11 Service on Pinebook Pro" /></p>
<p>That's the X11 Service... A <strong>2.2 MB</strong> executable named <strong><code>Xorg</code></strong>. </p>
<p>The X11 Service controls the rendering of Linux apps (as well as the keyboard and mouse input) like this...</p>
<p><img src="https://lupyuen.github.io/images/wayland-x11.png" alt="X11 Architecture" /></p>
<p>(Adapted from <a href="https://wayland.freedesktop.org/architecture.html">&quot;Wayland Architecture&quot;</a>)</p>
<ol>
<li>
<p>At the top we have the Linux programs running on our Linux machine: <strong>Terminal, Editor, Web Browser</strong>.</p>
<p>Each program renders its graphical display and transmits the raw graphics to the X11 Service (via a local TCP socket).</p>
</li>
<li>
<p>X11 Service forwards the rendered graphics to the <a href="https://en.wikipedia.org/wiki/Compositing_window_manager"><strong>Window Manager / Compositor</strong></a>.</p>
<p>The Window Manager / Compositor is provided by the <strong>Desktop Environment</strong>: Xfce, KDE, Gnome, ...</p>
</li>
<li>
<p>The Window Manager / Compositor wraps the rendered graphics into Display Windows and <strong>&quot;decorates&quot;</strong> them with scrollbars, title bar and minimise / maximise / close buttons.</p>
<p>The Window Manager / Compositor then draws the Display Windows into a <strong>Screen Buffer</strong> according to their screen coordinates.</p>
</li>
<li>
<p>The Screen Buffer is rendered to our screen by the X11 Service, talking to the <strong>Linux Display Driver</strong>.</p>
</li>
<li>
<p>Any <strong>keyboard and mouse input</strong> is captured by the X11 Service, and forwarded to the programs.</p>
</li>
</ol>
<p><em>Why is X11 so complex? So many hops?</em></p>
<p>Because X11 was designed for Distributed Computing Systems.</p>
<p>Here's how I used (abused?) X11R4 at <a href="http://srg.cs.illinois.edu/">UIUC Systems Research Group</a> way back in 1990 (30 years ago!)...</p>
<p><img src="https://lupyuen.github.io/images/wayland-uiuc.png" alt="Distributed X11 System" /></p>
<p>Thankfully things are a lot simpler now, lemme explain...</p>
<h1 id="wayland-on-ubuntu-touch" class="section-header"><a href="#wayland-on-ubuntu-touch">2 Wayland on Ubuntu Touch</a></h1>
<p><em>Do we need overlapping or tiled windows on PinePhone?</em></p>
<p><em>Do we need to need to decorate PinePhone windows with a title bar and minimise / maximise / close buttons?</em></p>
<p><em>Do we even need any windows on PinePhone?</em></p>
<p>No! Because each PinePhone app takes control of the entire screen!</p>
<p>PinePhone uses a simpler Graphical Display Service: the <a href="https://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Wayland_compositors"><strong>Wayland Compositor</strong></a>.</p>
<p>Let's hunt for the Wayland Compositor on PinePhone...</p>
<p><img src="https://lupyuen.github.io/images/wayland-compositor.png" alt="Wayland Compositor on PinePhone" /></p>
<p>That's the Wayland Compositor... A <strong>262 KB</strong> executable named <code>unity-system-compositor</code>. </p>
<p><em>Compare that with the 2.2 MB X11 Server on Pinebook Pro!</em></p>
<p>Here's how the Wayland Compositor controls apps and touchscreen input on PinePhone with Ubuntu Touch...</p>
<p><img src="https://lupyuen.github.io/images/wayland-arch.png" alt="Wayland Architecture" /></p>
<p>(Adapted from <a href="https://wayland.freedesktop.org/architecture.html">&quot;Wayland Architecture&quot;</a> and <a href="https://en.wikipedia.org/wiki/EGL_(API)">&quot;EGL API&quot;</a>)</p>
<ol>
<li>
<p>At the top we have the apps running on our phone: <strong>Terminal, Editor, Web Browser</strong>.</p>
<p>Since each app runs fullscreen, only the active app will be rendered.</p>
<p>When then app starts, it queries the <strong>Wayland Compositor</strong> for the graphical display interfaces available. (They talk via a <a href="https://en.wikipedia.org/wiki/Unix_file_types#Socket">Linux socket file</a>: <code>/run/user/32011/wayland-0</code>)</p>
</li>
<li>
<p>Wayland Compositor returns the <a href="https://en.wikipedia.org/wiki/EGL_(API)"><strong>EGL Interface</strong></a> to the app.</p>
</li>
<li>
<p>App calls the EGL Interface to <a href="https://en.wikipedia.org/wiki/OpenGL"><strong>render OpenGL graphics</strong></a> directly to the <strong>Linux Display Driver</strong>.</p>
</li>
<li>
<p>Linux Display Driver forwards the OpenGL rendering commands to the <a href="https://en.wikipedia.org/wiki/Graphics_processing_unit"><strong>GPU to update the screen</strong></a>.</p>
</li>
<li>
<p>Any <strong>touchscreen input</strong> is captured by the Wayland Compositor, and forwarded to the active app.</p>
</li>
</ol>
<p>Wayland looks so much simpler and faster than X11!</p>
<p><em>Wayland is designed for OpenGL and GPUs?</em></p>
<p>Yes! And I lied about Wayland being New Underwear... Wayland is not really that New!</p>
<p>Wayland was first released in 2008 (<a href="https://en.wikipedia.org/wiki/Wayland_(display_server_protocol)">11 years ago</a>)... Yet it was designed around OpenGL and GPUs, the same tech that powers our beautiful games today. (<a href="https://youtu.be/DNBk9hnPkTY">And websites too</a>)</p>
<p>Read on to learn how to render our own OpenGL graphics with Wayland and Ubuntu Touch on PinePhone...</p>
<p><img src="https://lupyuen.github.io/images/wayland-egl.jpg" alt="Rendering a yellow rectangle with Wayland and OpenGL on PinePhone" /></p>
<p><em>Rendering a yellow rectangle with Wayland and OpenGL on PinePhone</em></p>
<h1 id="render-opengl-graphics-with-wayland" class="section-header"><a href="#render-opengl-graphics-with-wayland">3 Render OpenGL Graphics with Wayland</a></h1>
<p>Here's the function that calls OpenGL to render the yellow box above: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c#L44-L60"><code>pinephone-mir/egl.c</code></a></p>
<pre><code class="language-c">/// Render the OpenGL ES2 display
static void render_display() {
    //  Fill the rectangular region with yellow
    glClearColor(
        1.0,  //  Red
        1.0,  //  Green
        0.0,  //  Blue
        1.0   //  Alpha
    );
    glClear(GL_COLOR_BUFFER_BIT);

    // Render now
    glFlush();
}
</code></pre>
<p><code>render_display()</code> looks exactly like normal OpenGL, and it works on PinePhone with Wayland! (Thanks to Ubuntu Touch)</p>
<p>Two things to note...</p>
<ol>
<li>
<p>PinePhone supports a popular subset of OpenGL, known as <a href="https://en.wikipedia.org/wiki/OpenGL_ES"><strong>OpenGL for Embedded Systems</strong></a> Version 2.0.</p>
<p>OpenGL ES is optimised for Embedded Devices. It's used by many mobile and console games today.</p>
</li>
<li>
<p>To render OpenGL ES graphics, we need to get the <strong>OpenGL ES Context</strong> and <strong>Window Surface</strong> from Wayland</p>
</li>
</ol>
<p>Before calling <code>render_display()</code>, we fetch the OpenGL Window Surface from Wayland like so: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c#L167-L189"><code>pinephone-mir/egl.c</code></a></p>
<pre><code class="language-c">/// Dimensions of the OpenGL region to be rendered
static int WIDTH  = 480;
static int HEIGHT = 360;

static struct wl_egl_window *egl_window;  //  Wayland EGL Window
static EGLSurface egl_surface;            //  EGL Surface

//  Create the EGL Window and render OpenGL graphics
static void create_window(void) {
    //  Create an EGL Window from a Wayland Surface 
    egl_window = wl_egl_window_create(surface, WIDTH, HEIGHT);
    assert(egl_window != EGL_NO_SURFACE);  //  Failed to create OpenGL Window

    //  Create an OpenGL Window Surface for rendering
    egl_surface = eglCreateWindowSurface(egl_display, egl_conf,
        egl_window, NULL);
    assert(egl_surface != NULL);  //  Failed to create OpenGL Window Surface

    //  Set the current rendering surface
    EGLBoolean madeCurrent = eglMakeCurrent(egl_display, egl_surface,
        egl_surface, egl_context);
    assert(madeCurrent);  //  Failed to set rendering surface

    //  Render the display
    render_display();

    //  Swap the display buffers to make the display visible
    EGLBoolean swappedBuffers = eglSwapBuffers(egl_display, egl_surface);
    assert(swappedBuffers);  //  Failed to swap display buffers
}
</code></pre>
<p>Functions named <code>wl_egl_...</code> are provided by the Wayland EGL Interface.  Functions named <code>egl...</code> come from the cross-platform <a href="https://docs.mesa3d.org/egl.html#:%7E:text=The%20main%20library%20(%20libEGL%20)%20is,directly%20dispatched%20to%20the%20drivers.">Mesa 3D Graphics Library</a>.</p>
<p><em>EGL vs OpenGL... What's the difference?</em></p>
<p>In Wayland, EGL is the Enabler for OpenGL. </p>
<p>Wayland only understands EGL and it will gladly hand us EGL objects... But it's up to us to transform EGL into OpenGL for rendering.</p>
<p>Thus in the code above, we take a Wayland Surface <code>surface</code> and transform it into an EGL Window <code>egl_window</code>...</p>
<pre><code class="language-c">//  Create an EGL Window from a Wayland Surface 
egl_window = wl_egl_window_create(surface, WIDTH, HEIGHT);
</code></pre>
<p>Then we create an OpenGL Window Surface <code>egl_surface</code> from that EGL Window...</p>
<pre><code class="language-c">//  Create an OpenGL Window Surface for rendering
egl_surface = eglCreateWindowSurface(egl_display, egl_conf,
    egl_window, NULL);
</code></pre>
<p>And we begin the OpenGL rendering...</p>
<pre><code class="language-c">//  Set the current rendering surface
eglMakeCurrent(egl_display, egl_surface,
    egl_surface, egl_context);

//  Render the display
render_display();

//  Swap the display buffers to make the display visible
eglSwapBuffers(egl_display, egl_surface);
</code></pre>
<p>Here's how we create a Wayland Region for OpenGL rendering: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c#L103-L112"><code>pinephone-mir/egl.c</code></a></p>
<pre><code class="language-c">static struct wl_region *region;  //  Wayland Region

//  Create the Wayland Region for rendering OpenGL graphics
static void create_opaque_region(void) {
    //  Create a Wayland Region
    region = wl_compositor_create_region(compositor);
    assert(region != NULL);  //  Failed to create EGL Region

    //  Set the dimensions of the Wayland Region
    wl_region_add(region, 0, 0, WIDTH, HEIGHT);

    //  Add the Region to the Wayland Surface
    wl_surface_set_opaque_region(surface, region);
}
</code></pre>
<p>To learn more about EGL, check out <a href="https://jan.newmarch.name/Wayland/EGL/">&quot;Programming Wayland Clients&quot;</a></p>
<p>The Wayland EGL code in this article was adapted from that document.</p>
<h1 id="get-egl-context-from-wayland" class="section-header"><a href="#get-egl-context-from-wayland">4 Get EGL Context from Wayland</a></h1>
<p>Earlier in <code>create_window()</code> we called an <strong>EGL Context</strong> <code>egl_context</code> to render OpenGL graphics.</p>
<p>Here's how we get the EGL Context: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c#L113-L165"><code>pinephone-mir/egl.c</code></a></p>
<pre><code class="language-c">/// Wayland EGL Interfaces for OpenGL Rendering
static EGLDisplay egl_display;  //  EGL Display
static EGLConfig  egl_conf;     //  EGL Configuration
static EGLContext egl_context;  //  EGL Context

//  Create the EGL Context for rendering OpenGL graphics
static void init_egl(void) {
    //  Attributes for our EGL Display
    EGLint config_attribs[] = {
        EGL_SURFACE_TYPE,    EGL_WINDOW_BIT,
        EGL_RED_SIZE,        8,
        EGL_GREEN_SIZE,      8,
        EGL_BLUE_SIZE,       8,
        EGL_RENDERABLE_TYPE, EGL_OPENGL_ES2_BIT,
        EGL_NONE
    };
    static const EGLint context_attribs[] = {
        EGL_CONTEXT_CLIENT_VERSION, 2,
        EGL_NONE
    };

    //  Get the EGL Display
    egl_display = eglGetDisplay((EGLNativeDisplayType) display);
    assert(egl_display != EGL_NO_DISPLAY);  //  Failed to get EGL Display

    //  Init the EGL Display
    EGLint major, minor;
    EGLBoolean egl_init = eglInitialize(egl_display, &amp;major, &amp;minor);
    assert(egl_init);  //  Failed to init EGL Display

    //  Get the EGL Configurations
    EGLint count, n;
    eglGetConfigs(egl_display, NULL, 0, &amp;count);
    EGLConfig *configs = calloc(count, sizeof *configs);
    eglChooseConfig(egl_display, config_attribs,
        configs, count, &amp;n);

    //  Choose the first EGL Configuration
    for (int i = 0; i &lt; n; i++) {
        EGLint size;
        eglGetConfigAttrib(egl_display, configs[i], EGL_BUFFER_SIZE, &amp;size);
        eglGetConfigAttrib(egl_display, configs[i], EGL_RED_SIZE, &amp;size);
        egl_conf = configs[i];
        break;
    }
    assert(egl_conf != NULL);  //  Failed to get EGL Configuration

    //  Create the EGL Context based on the EGL Display and Configuration
    egl_context = eglCreateContext(egl_display, egl_conf,
        EGL_NO_CONTEXT, context_attribs);
    assert(egl_context != NULL);  //  Failed to create EGL Context
}
</code></pre>
<p><a href="https://jan.newmarch.name/Wayland/EGL/">More about EGL</a></p>
<p>The above code in <code>init_egl()</code> creates the EGL Context. </p>
<p>We call <code>init_egl()</code> in our Main Function like so: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c#L64-L98"><code>pinephone-mir/egl.c</code></a></p>
<pre><code class="language-c">/// Wayland Interfaces
static struct wl_surface       *surface;       //  Wayland Surface
static struct wl_shell_surface *shell_surface; //  Wayland Shell Surface

/// Connect to Wayland Compositor and render OpenGL graphics
int main(int argc, char **argv) {
    //  Get interfaces for Wayland Compositor and Wayland Shell
    get_server_references();
    assert(display != NULL);     //  Failed to get Wayland Display
    assert(compositor != NULL);  //  Failed to get Wayland Compositor
    assert(shell != NULL);       //  Failed to get Wayland Shell

    //  Create a Wayland Surface for rendering our app
    surface = wl_compositor_create_surface(compositor);
    assert(surface != NULL);  //  Failed to create Wayland Surface

    //  Get the Wayland Shell Surface for rendering our app window
    shell_surface = wl_shell_get_shell_surface(shell, surface);
    assert(shell_surface != NULL);

    //  Set the Shell Surface as top level
    wl_shell_surface_set_toplevel(shell_surface);

    //  Create the Wayland Region for rendering OpenGL graphics
    create_opaque_region();

    //  Create the EGL Context for rendering OpenGL graphics
    init_egl();

    //  Create the EGL Window and render OpenGL graphics
    create_window();

    //  Handle all Wayland Events in the Event Loop
    while (wl_display_dispatch(display) != -1) {}

    //  Disconnect from the Wayland Display
    wl_display_disconnect(display);
    return 0;
}
</code></pre>
<p>In all Wayland apps, the <code>main()</code> function follows the same steps...</p>
<ol>
<li>
<p>Fetch the <strong>Wayland Compositor</strong> and <strong>Wayland Shell</strong> from the <strong>Wayland Registry</strong>...</p>
<pre><code class="language-c">//  Get interfaces for Wayland Compositor and Wayland Shell
get_server_references();
</code></pre>
<p>We'll talk about <code>get_server_references()</code> and the Wayland Registry in a while.</p>
</li>
<li>
<p>Every Wayland App needs a <strong>Wayland Surface</strong> (screen buffer) for displaying the app...</p>
<pre><code class="language-c">//  Create a Wayland Surface for rendering our app
surface = wl_compositor_create_surface(compositor);
</code></pre>
</li>
<li>
<p>Create a <strong>Wayland Shell Surface</strong> (app window) for rendering our app...</p>
<pre><code class="language-c">//  Get the Wayland Shell Surface for rendering our app window
shell_surface = wl_shell_get_shell_surface(shell, surface);
</code></pre>
</li>
<li>
<p>Set the Shell Surface as the <strong>Top Level</strong> window for our app...</p>
<pre><code class="language-c">//  Set the Shell Surface as top level
wl_shell_surface_set_toplevel(shell_surface);
</code></pre>
</li>
<li>
<p>This part is specific to OpenGL apps...</p>
<p>Earlier we have seen <code>create_opaque_region()</code>, <code>init_egl()</code> and <code>create_window()</code>. We call them to create the Wayland Region, EGL Context and EGL Window, and to render the OpenGL graphics.</p>
<pre><code class="language-c">//  Create the Wayland Region for rendering OpenGL graphics
create_opaque_region();
//  Create the EGL Context for rendering OpenGL graphics
init_egl();
//  Create the EGL Window and render OpenGL graphics
create_window();
</code></pre>
</li>
<li>
<p>Every Wayland App needs an <strong>Event Loop</strong> for handling Wayland Events...</p>
<pre><code class="language-c">//  Handle all Wayland Events in the Event Loop
while (wl_display_dispatch(display) != -1) {}
</code></pre>
</li>
<li>
<p>When our app terminates, we disconnect the Wayland Display...</p>
<pre><code class="language-c">//  Disconnect from the Wayland Display
wl_display_disconnect(display);
</code></pre>
</li>
</ol>
<p>Now let's build and test the app on our Linux development machine. (We'll run it on PinePhone later)</p>
<h1 id="build-and-test-wayland-app-on-linux" class="section-header"><a href="#build-and-test-wayland-app-on-linux">5 Build and Test Wayland App on Linux</a></h1>
<p>Now that we have created a <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c">simple Wayland app</a> that renders OpenGL graphics... Let's build it!</p>
<p>Building a Wayland app is refreshingly simple (if you're used to GDK, Qt and SDL).</p>
<p>Here's how we build the Wayland app in <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c"><code>egl.c</code></a> on a Linux machine (that has Wayland, MESA EGL and OpenGL ES2 libraries installed)...</p>
<pre><code class="language-bash"># Build the Wayland EGL app
gcc \
    -g \
    -o egl \
    egl.c \
    -Wl,-Map=egl.map \
    -L/usr/lib/aarch64-linux-gnu/mesa-egl \
    -lwayland-client \
    -lwayland-server \
    -lwayland-egl \
    -lEGL \
    -lGLESv2
</code></pre>
<p>This produces the executable app <code>egl</code>. Run the <code>egl</code> app on our Linux machine like so...</p>
<pre><code class="language-bash"># Install Weston Wayland Compositor...
# For Arch Linux and Manjaro:
sudo pacman -S weston

# For Other Distros:
# Check https://github.com/wayland-project/weston

# Start the Weston Wayland Compositor on our computer with the PinePhone screen dimensions
weston --width=720 --height=1398 &amp;

# Run the Wayland EGL app
./egl
</code></pre>
<p>This uses the <a href="https://github.com/wayland-project/weston"><strong>Weston Compositor</strong></a>, the reference implementation of the Wayland Compositor that runs on X11.</p>
<p>We'll see this Inception-like window within a window...</p>
<p><img src="https://lupyuen.github.io/images/wayland-westonegl.png" alt="EGL App running with Wayland Weston Compositor on Pinebook Pro" /></p>
<p>We learn in a while how to build and run the app on PinePhone.</p>
<h1 id="fetch-wayland-interfaces" class="section-header"><a href="#fetch-wayland-interfaces">6 Fetch Wayland Interfaces</a></h1>
<p>Earlier we used the Wayland Compositor and the Wayland Shell in our app...</p>
<ol>
<li>
<p><strong>Wayland Compositor</strong> (<code>compositor</code>): Manages the screen buffer used by apps</p>
</li>
<li>
<p><strong>Wayland Shell</strong> (<code>shell</code>): Manages the app windows</p>
</li>
</ol>
<p>Here's how we fetch the two interfaces from Wayland: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c#L194-L251"><code>pinephone-mir/egl.c</code></a></p>
<pre><code class="language-c">/// Wayland Interfaces
static struct wl_display       *display;       //  Wayland Display
static struct wl_compositor    *compositor;    //  Wayland Compositor
static struct wl_shell         *shell;         //  Wayland Shell

/// Connect to Wayland Service and fetch the interfaces for Wayland Compositor and Wayland Shell
static void get_server_references(void) {
    //  Connect to the Wayland Service
    display = wl_display_connect(NULL);
    if (display == NULL) {
        fprintf(stderr, &quot;Failed to connect to display\n&quot;);
        exit(1);
    }

    //  Get the Wayland Registry
    struct wl_registry *registry = wl_display_get_registry(display);
    assert(registry != NULL);  //  Failed to get Wayland Registry

    //  Add Registry Callbacks to handle interfaces returned by Wayland Service
    wl_registry_add_listener(registry, &amp;registry_listener, NULL);

    //  Wait for Registry Callbacks to fetch Wayland Interfaces
    wl_display_dispatch(display);
    wl_display_roundtrip(display);

    //  We should have received interfaces for Wayland Compositor and Wayland Shell
    assert(compositor != NULL);  //  Failed to get Wayland Compositor
    assert(shell != NULL);       //  Failed to get Wayland Shell
}
</code></pre>
<p>What happens inside <code>get_server_references()</code>?</p>
<ol>
<li>
<p>The Wayland Compositor runs as a Linux Service that listens on a <a href="https://en.wikipedia.org/wiki/Unix_file_types#Socket">Linux Socket File</a>: <code>/run/user/32011/wayland-0</code> for PinePhone on Ubuntu Touch.</p>
<p>We connect to the <strong>Wayland Service</strong> like so...</p>
<pre><code class="language-c">//  Connect to the Wayland Service
display = wl_display_connect(NULL);
</code></pre>
<p>Remember that all functions named <code>wl_...</code> come from the Wayland Library.</p>
</li>
<li>
<p>To work with the Wayland Service, we fetch the Interfaces for the Wayland Compositor and Wayland Shell.</p>
<p><strong>Wayland Interfaces</strong> are defined in the <strong>Wayland Registry</strong>...</p>
<pre><code class="language-c">//  Get the Wayland Registry
struct wl_registry *registry = wl_display_get_registry(display);
</code></pre>
</li>
<li>
<p>To fetch the Compositor and Shell from the Wayland Registry, we add a <strong>Registry Listener</strong> (more about this later)...</p>
<pre><code class="language-c">//  Add Registry Callbacks to handle interfaces returned by Wayland Service
wl_registry_add_listener(registry, &amp;registry_listener, NULL);
</code></pre>
</li>
<li>
<p>Now we <strong>dispatch the Registry Listener request</strong> to the Wayland Service. (Remember that the Wayland Service operates on Linux Socket Messages)</p>
<pre><code class="language-c">//  Wait for Registry Callbacks to fetch Wayland Interfaces
wl_display_dispatch(display);
wl_display_roundtrip(display);
</code></pre>
</li>
</ol>
<p>And we'll get the <code>compositor</code> and <code>shell</code> objects populated from the Wayland Registry!</p>
<p>If you're curious, the Registry Listener works like this: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl.c#L194-L251"><code>pinephone-mir/egl.c</code></a></p>
<pre><code class="language-c">/// Callbacks for interfaces returned by Wayland Service
static const struct wl_registry_listener registry_listener = {
    global_registry_handler,
    global_registry_remover
};

/// Callback for interfaces returned by Wayland Service
static void global_registry_handler(void *data, struct wl_registry *registry, uint32_t id,
    const char *interface, uint32_t version) {
    printf(&quot;Got interface %s id %d\n&quot;, interface, id);

    if (strcmp(interface, &quot;wl_compositor&quot;) == 0) {
        //  Bind to Wayland Compositor Interface
        compositor = wl_registry_bind(registry, id,
            &amp;wl_compositor_interface,   //  Interface Type
            1);                         //  Interface Version
    } else if (strcmp(interface, &quot;wl_shell&quot;) == 0){
        //  Bind to Wayland Shell Interface
        shell = wl_registry_bind(registry, id,
            &amp;wl_shell_interface,        //  Interface Type
            1);                         //  Interface Version
    }
}
</code></pre>
<p><code>global_registry_handler()</code> is the Callback Function that will be triggered for every interface in the Wayland Registry. </p>
<p>The Wayland Service for Ubuntu Touch <code>unity-system-compositor</code> returns a whole bunch of interesting Wayland Interfaces (like <code>qt_windowmanager</code>). </p>
<p>But today we'll bind to the Compositor Interface named <code>wl_compositor</code> and Shell Interface named <code>wl_shell</code>.</p>
<p>And that's how we render a yellow rectangle with Wayland and OpenGL!</p>
<p>Let's move on to something more interesting: Rendering a simple bitmap texture...</p>
<p><img src="https://lupyuen.github.io/images/wayland-egl2.jpg" alt="Rendering a simple bitmap texture with Wayland and OpenGL on PinePhone" /></p>
<p><em>Rendering a simple bitmap texture with Wayland and OpenGL on PinePhone</em></p>
<h1 id="render-opengl-bitmap-texture-with-wayland" class="section-header"><a href="#render-opengl-bitmap-texture-with-wayland">7 Render OpenGL Bitmap Texture with Wayland</a></h1>
<p>The four boxes we see above are rendered from a magnified <strong>2-pixel by 2-pixel bitmap</strong>: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/texture.c#L30-L64"><code>pinephone-mir/texture.c</code></a></p>
<pre><code class="language-c">// 2x2 Image, 3 bytes per pixel (R, G, B)
GLubyte pixels[4 * 3] = {
    255, 0, 0,  // Red
    0, 255, 0,  // Green
    0, 0, 255,  // Blue
    255, 255, 0 // Yellow
};
</code></pre>
<p>We render the bitmap by creating an <strong>OpenGL Texture</strong>: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/texture.c#L30-L64"><code>pinephone-mir/texture.c</code></a></p>
<pre><code class="language-c">// Create a simple 2x2 texture image with four different colors
GLuint CreateSimpleTexture2D() {
    // Texture object handle
    GLuint textureId;

    // 2x2 Image, 3 bytes per pixel (R, G, B)
    GLubyte pixels[4 * 3] = {
        255, 0, 0,  // Red
        0, 255, 0,  // Green
        0, 0, 255,  // Blue
        255, 255, 0 // Yellow
    };

    // Use tightly packed data
    glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

    // Generate a texture object
    glGenTextures(1, &amp;textureId);

    // Bind the texture object
    glBindTexture(GL_TEXTURE_2D, textureId);

    // Load the texture
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 2, 2, 0, GL_RGB, GL_UNSIGNED_BYTE, pixels);

    // Set the filtering mode
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
    return textureId;
}
</code></pre>
<p>(Not the most efficient way to render a bitmap... But let's try this and test drive PinePhone's GPU!)</p>
<p>This is the usual way we create an OpenGL Texture, as explained in <a href="http://www.opengles-book.com/">&quot;OpenGL® ES 3.0 Programming Guide&quot;</a>.</p>
<p>Here comes the tricky part... Before rendering the OpenGL Texture, we need to program the <strong>GPU Shaders</strong> on PinePhone with a C-like language: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/texture.c#L66-L93"><code>pinephone-mir/texture.c</code></a></p>
<pre><code class="language-c">// Initialize the shader and program object
int Init(ESContext *esContext) {
    esContext-&gt;userData = malloc(sizeof(UserData));
    UserData *userData = esContext-&gt;userData;
    GLbyte vShaderStr[] =
        &quot;attribute vec4 a_position;   \n&quot;
        &quot;attribute vec2 a_texCoord;   \n&quot;
        &quot;varying vec2 v_texCoord;     \n&quot;
        &quot;void main()                  \n&quot;
        &quot;{                            \n&quot;
        &quot;   gl_Position = a_position; \n&quot;
        &quot;   v_texCoord = a_texCoord;  \n&quot;
        &quot;}                            \n&quot;;

    GLbyte fShaderStr[] =
        &quot;precision mediump float;                            \n&quot;
        &quot;varying vec2 v_texCoord;                            \n&quot;
        &quot;uniform sampler2D s_texture;                        \n&quot;
        &quot;void main()                                         \n&quot;
        &quot;{                                                   \n&quot;
        &quot;  gl_FragColor = texture2D( s_texture, v_texCoord );\n&quot;
        &quot;}                                                   \n&quot;;

    // Load the shaders and get a linked program object
    userData-&gt;programObject = esLoadProgram(vShaderStr, fShaderStr);
    ...
</code></pre>
<p>(Yep a C program within a C program... Inception!)</p>
<p><code>esLoadProgram()</code> is defined in <a href="https://github.com/lupyuen/pinephone-mir/blob/master/shader.c"><code>pinephone-mir/shader.c</code></a></p>
<p>We're now talking to PinePhone's GPU, which is so low-level that it understand only Triangles, not Rectangles.</p>
<p>Hence to render the OpenGL Texture, we map the Rectangular Texture onto two Triangles and render them: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/texture.c#L109-L130"><code>pinephone-mir/texture.c</code></a></p>
<pre><code class="language-c">// Draw a triangle using the shader pair created in Init()
void Draw(ESContext *esContext) {
    GLfloat vVertices[] = {
        -0.5f,   0.5f,  0.0f,  // Position 0
         0.0f,   0.0f,         // TexCoord 0
        -0.5f,  -0.5f,  0.0f,  // Position 1
         0.0f,   1.0f,         // TexCoord 1
         0.5f,  -0.5f,  0.0f,  // Position 2
         1.0f,   1.0f,         // TexCoord 2
         0.5f,   0.5f,  0.0f,  // Position 3
         1.0f,   0.0f          // TexCoord 3
    };
    GLushort indices[] = {
        0, 1, 2,  //  First Triangle
        0, 2, 3   //  Second Triangle
    };
    ...
    //  Draw the 6 vertices as 2 triangles
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, indices);
}
</code></pre>
<p>(Yes the math is starting to hurt... But that's the end of it!)</p>
<p>Finally we connect the above code to render the four colour boxes on PinePhone, thanks to Wayland and OpenGL: <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl2.c#L51-L67">pinephone-mir/egl2.c</a></p>
<pre><code class="language-c">/// Render the OpenGL ES2 display
static void render_display() {
    //  Create the texture context
    static ESContext esContext;
    esInitContext ( &amp;esContext );
    esContext.width  = WIDTH;
    esContext.height = HEIGHT;

    //  Draw the texture
    Init(&amp;esContext);
    Draw(&amp;esContext);

    //  Render now
    glFlush();
}
</code></pre>
<p>And that's <a href="https://github.com/lupyuen/pinephone-mir/blob/master/egl2.c">our Wayland App</a> that renders a simple OpenGL Bitmap Texture!</p>
<p>The OpenGL Texture code in this article was adapted from <a href="https://github.com/danginsburg/opengles-book-samples">&quot;OpenGL® ES 2.0 Programming Guide&quot;</a></p>
<p>Let's head on towards greatness and something really useful: Graphical User Interfaces...</p>
<p><img src="https://lupyuen.github.io/images/wayland-button.jpg" alt="Button rendered with LVGL and Wayland on PinePhone" /></p>
<p><em>Button rendered with LVGL and Wayland on PinePhone</em></p>
<h1 id="lvgl-toolkit-for-graphical-user-interfaces" class="section-header"><a href="#lvgl-toolkit-for-graphical-user-interfaces">8 LVGL Toolkit for Graphical User Interfaces</a></h1>
<p>Now that we can render bitmaps on PinePhone, let's think...</p>
<p><em>How would we render a simple Graphical User Interface (GUI) on PinePhone, like the button above?</em></p>
<p>Why don't we use a simple GUI Toolkit like <a href="https://lvgl.io/"><strong>LVGL</strong></a>? (Formerly LittleVGL)</p>
<p>Here's how we call the LVGL library to render that button: <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lvgl.c#L54-L64"><code>lvgl-wayland/wayland/lvgl.c</code></a></p>
<pre><code class="language-c">#include &quot;../lvgl.h&quot;

/// Render a Button Widget and a Label Widget
static void render_widgets(void) {
    lv_obj_t * btn = lv_btn_create(lv_scr_act(), NULL);     //  Add a button the current screen
    lv_obj_set_pos(btn, 10, 10);                            //  Set its position
    lv_obj_set_size(btn, 120, 50);                          //  Set its size

    lv_obj_t * label = lv_label_create(btn, NULL);          //  Add a label to the button
    lv_label_set_text(label, &quot;Button&quot;);                     //  Set the labels text
}
</code></pre>
<p><em>Easy peasy!</em></p>
<p>LVGL is a simple C toolkit designed for Embedded Devices, so it needs <strong>very little memory and processing power</strong>.  LVGL is <strong>self-contained</strong>... Fonts and icons are bundled into the LVGL library.</p>
<p>It's used on <a href="https://github.com/JF002/Pinetime"><strong>PineTime Smart Watch</strong></a> to render watch faces.</p>
<p><em>LVGL doesn't run on Wayland yet... But we'll fix that!</em></p>
<p>Remember how we rendered a simple 2-pixel by 2-pixel bitmap by creating an OpenGL Texture with <code>CreateSimpleTexture2D()</code>?</p>
<p>Let's extend that bitmap to cover the entire PinePhone screen: 720 pixels by 1398 pixels.</p>
<p>And we create the OpenGL Texture for the entire PinePhone screen like so: <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/texture.c#L38-L72"><code>lvgl-wayland/wayland/texture.c</code></a></p>
<pre><code class="language-c">///  PinePhone Screen Resolution, defined in lv_conf.h
#define LV_HOR_RES_MAX          (720)
#define LV_VER_RES_MAX          (1398)
#define LV_SCALE_RES            1

///  Screen buffer 
#define BYTES_PER_PIXEL 3
GLubyte pixels[LV_HOR_RES_MAX * LV_VER_RES_MAX * BYTES_PER_PIXEL];

/// Create an OpenGL Texture for the screen buffer
GLuint CreateTexture(void) {
    GLuint texId;
    glGenTextures ( 1, &amp;texId );
    glBindTexture ( GL_TEXTURE_2D, texId );

    glTexImage2D (
        GL_TEXTURE_2D, 
        0,  //  Level
        GL_RGB, 
        LV_HOR_RES_MAX,  //  Width
        LV_VER_RES_MAX,  //  Height 
        0,  //  Format 
        GL_RGB, 
        GL_UNSIGNED_BYTE, 
        pixels 
    );
    glTexParameteri ( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );
    glTexParameteri ( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );
    glTexParameteri ( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );
    glTexParameteri ( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );
    return texId;
}
</code></pre>
<p><code>pixels</code> is the screen buffer that will contain the pixels for our rendered UI controls, like our button.</p>
<p>We'll tell LVGL to render into <code>pixels</code> like so: <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/texture.c#L38-L72"><code>lvgl-wayland/wayland/texture.c</code></a></p>
<pre><code class="language-c">/// Set the colour of a pixel in the screen buffer
void put_px(uint16_t x, uint16_t y, uint8_t r, uint8_t g, uint8_t b, uint8_t a) {
    assert(x &gt;= 0); assert(x &lt; LV_HOR_RES_MAX);
    assert(y &gt;= 0); assert(y &lt; LV_VER_RES_MAX);
    int i = (y * LV_HOR_RES_MAX * BYTES_PER_PIXEL) + (x * BYTES_PER_PIXEL);
    pixels[i++] = r;  //  Red
    pixels[i++] = g;  //  Green
    pixels[i++] = b;  //  Blue
}
</code></pre>
<p>(Simplistic, not efficient though)</p>
<p>We'll render the OpenGL Texture the same way as before: <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lvgl.c#L65-L93"><code>lvgl-wayland/wayland/lvgl.c</code></a></p>
<pre><code class="language-c">/// Render the OpenGL ES2 display
static void render_display() {
    //  This part is new...

    //  Init the LVGL display
    lv_init();
    lv_port_disp_init();

    //  Create the LVGL widgets: Button and label
    render_widgets();

    //  Render the LVGL widgets
    lv_task_handler();

    //  This part is the same as before...

    //  Create the texture context
    static ESContext esContext;
    esInitContext ( &amp;esContext );
    esContext.width  = WIDTH;
    esContext.height = HEIGHT;

    //   Draw the texture
    Init(&amp;esContext);
    Draw(&amp;esContext);

    //  Render now
    glFlush();
}
</code></pre>
<p>But now we have injected the calls to the LVGL library...</p>
<ol>
<li>
<p><a href="https://docs.lvgl.io/latest/en/html/widgets/obj.html?highlight=lv_init#_CPPv47lv_initv"><strong><code>lv_init()</code></strong></a>: Initialise the LVGL library</p>
</li>
<li>
<p><a href="https://github.com/lupyuen/lvgl-wayland/blob/72b0273d6c609ecb51142ee400f545116ca0ecd9/wayland/lv_port_disp.c#L48-L126"><strong><code>lv_port_disp_init()</code></strong></a>: Initialise our display</p>
</li>
<li>
<p><a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lvgl.c#L54-L64"><strong><code>render_widgets()</code></strong></a>: Calls the LVGL library to create two UI controls: a Button and a Label</p>
</li>
<li>
<p><a href="https://docs.lvgl.io/latest/en/html/porting/task-handler.html?highlight=lv_task_handler"><strong><code>lv_task_handler()</code></strong></a>: Let LVGL render the UI controls into our screen buffer</p>
</li>
</ol>
<p>Now let's tweak the LVGL library to render UI controls into our screen buffer <code>pixels</code></p>
<h1 id="port-lvgl-to-wayland" class="section-header"><a href="#port-lvgl-to-wayland">9 Port LVGL to Wayland</a></h1>
<p>Porting LVGL to Wayland and Ubuntu Touch is straightforward.</p>
<p>According to the <a href="https://docs.lvgl.io/latest/en/html/porting/display.html">LVGL Porting Doc</a>, we need to code a Flush Callback Function <code>disp_flush()</code> that will be called by LVGL to render UI controls to the screen buffer.</p>
<p>Here's our implementation for Wayland: <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lv_port_disp.c#L142-L167"><code>lvgl-wayland/wayland/lv_port_disp.c</code></a></p>
<pre><code class="language-c">//  Flush the content of the internal buffer to the specific area on the display
//  You can use DMA or any hardware acceleration to do this operation in the background but
//  'lv_disp_flush_ready()' has to be called when finished.
static void disp_flush(lv_disp_drv_t * disp_drv, const lv_area_t * area, lv_color_t * color_p) {
    //  The most simple case (but also the slowest) to put all pixels to the screen one-by-one
    for(int32_t y = area-&gt;y1; y &lt;= area-&gt;y2; y++) {
        for(int32_t x = area-&gt;x1; x &lt;= area-&gt;x2; x++) {
            //  Put a pixel to the screen buffer
            put_px(x, y, 
                color_p-&gt;ch.red, 
                color_p-&gt;ch.green, 
                color_p-&gt;ch.blue, 
                0xff);
            color_p++;
        }
    }
    //  Inform the graphics library that we are ready with the flushing
    lv_disp_flush_ready(disp_drv);
}
</code></pre>
<p>We've seen earlier that <code>put_px()</code> draws pixels in the simplest way possible.  Eventually we should use PinePhone's GPU for rendering LVGL controls, by implementing the <a href="https://docs.lvgl.io/latest/en/html/porting/display.html#display-driver">LVGL GPU Callbacks</a>.</p>
<p>Light and Dark Themes are provided by LVGL. To select the default theme just edit <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/lv_conf.h#L444-L446"><code>lvgl-wayland/lv_conf.h</code></a></p>
<p>Here's Dark Theme...</p>
<pre><code class="language-c">//  For Dark Theme...
#define LV_THEME_DEFAULT_FLAG LV_THEME_MATERIAL_FLAG_DARK
</code></pre>
<p><img src="https://lupyuen.github.io/images/wayland-dark.jpg" alt="LVGL Dark Theme with Wayland on PinePhone" /></p>
<p>And Light Theme...</p>
<pre><code class="language-c">//  For Light Theme...
#define LV_THEME_DEFAULT_FLAG LV_THEME_MATERIAL_FLAG_LIGHT
</code></pre>
<p><img src="https://lupyuen.github.io/images/wayland-light.jpg" alt="LVGL Light Theme with Wayland on PinePhone" /></p>
<p>The screens above were rendered by updating one line in <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lvgl.c#L65-L76"><code>lvgl-wayland/wayland/lvgl.c</code></a>...</p>
<pre><code class="language-c">/// Render the OpenGL ES2 display
static void render_display() {
    //  Init the LVGL display
    lv_init();
    lv_port_disp_init();

    //  Create the LVGL widgets
    lv_demo_widgets();  //  Previously render_widgets()
</code></pre>
<p><code>lv_demo_widgets()</code> comes from <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/demo/lv_demo_widgets.c"><code>lvgl-wayland/demo/lv_demo_widgets.c</code></a></p>
<p><em>What about Touch Input in LVGL for Ubuntu Touch?</em></p>
<p>We haven't handled Touch Input yet... Lemme know if you're keen to help!</p>
<p><em>Do we really have to code LVGL Apps for PinePhone in C?</em></p>
<p>Rust is supported too! </p>
<p>We may write LVGL Apps for PinePhone in Rust by calling the <a href="https://github.com/rafaelcaricio/lvgl-rs"><strong><code>lvgl-rs</code> Rust Wrapper for LVGL</strong></a> by <a href="https://github.com/rafaelcaricio"><strong>Rafael Carício</strong></a>.</p>
<p>(Fun Fact: <code>lvgl-rs</code> was <a href="https://lupyuen.github.io/PineTime-apps/articles/watch_face">originally created</a> for PineTime Smart Watch... Today it's used by <a href="https://twitter.com/rafaelcaricio/status/1271886471260184577?s=20">Rust on PlayStation Portable</a> too!)</p>
<p><img src="https://lupyuen.github.io/images/wayland-size.jpg" alt="Size of LVGL Demo App on PinePhone with Ubuntu Touch" /></p>
<p><em>How small is LVGL on PinePhone with Ubuntu Touch?</em></p>
<p><strong>1.5 MB</strong> for the Light / Dark Theme LVGL Demo App above.</p>
<p>Not that big, considering that the font, icons and debugging symbols are bundled inside.</p>
<p><em>How does LVGL compare with Qt, GTK and SDL on PinePhone with Ubuntu Touch?</em></p>
<p>Qt is the only officially supported App Toolkit on Ubuntu Touch. </p>
<p>GTK and SDL are supposed to work on Wayland... But I couldn't get them to work on Ubuntu Touch. </p>
<p>(Probably because legacy X11 compatibility is missing from Ubuntu Touch, i.e. XWayland)</p>
<p>I applaud the maintainers of X11, Qt, GTK and SDL because every new release needs to support so many legacy features. Kudos!</p>
<p>But what if we could start from scratch, drop the legacy stuff, and build a simpler UI toolkit for Wayland?</p>
<p><em>LVGL is the experiment that we're undertaking today!</em></p>
<h1 id="build-lvgl-on-pinephone-with-ubuntu-touch" class="section-header"><a href="#build-lvgl-on-pinephone-with-ubuntu-touch">10 Build LVGL on PinePhone with Ubuntu Touch</a></h1>
<p>Follow these steps to build LVGL on PinePhone over SSH.</p>
<p>(If we haven't enabled SSH on PinePhone, check the &quot;Configure SSH on PinePhone&quot; instructions below)</p>
<p>Connect to PinePhone over SSH and enter these commands...</p>
<pre><code class="language-bash"># Make system folders writeable before installing any packages
sudo mount -o remount,rw /

# Install dev tools and GLES2 library
sudo apt install gcc gdb git make libgles2-mesa-dev

# Download the source code
cd ~
git clone https://github.com/lupyuen/lvgl-wayland
cd lvgl-wayland

# Build the app
make
</code></pre>
<p>This creates the executable <code>~/lvgl-wayland/wayland/lvgl</code></p>
<p><em>Can we just run <code>lvgl</code> from the Terminal Command Line?</em></p>
<p>Nope! Because Wayland and Ubuntu Touch are super-secure, thanks to <a href="https://en.wikipedia.org/wiki/AppArmor#:%7E:text=AppArmor%20(%22Application%20Armor%22),execute%20files%20on%20matching%20paths."><strong>AppArmor</strong></a>.</p>
<p>But there's a way to trick AppArmor into allowing <code>lvgl</code> to be launched (since we are Superuser). </p>
<p>Read on to learn how...</p>
<p><img src="https://lupyuen.github.io/images/wayland-apparmor.jpg" alt="Fighting AppArmor Security... Permission Denied!" /></p>
<p><em>Fighting AppArmor Security... Permission Denied!</em></p>
<h1 id="inject-lvgl-into-file-manager-app" class="section-header"><a href="#inject-lvgl-into-file-manager-app">11 Inject LVGL into File Manager App</a></h1>
<p>For rapid testing (and to work around AppArmor), we shall replace the <strong>File Manager</strong> app by our <code>lvgl</code> app because <strong>File Manager has no AppArmor restrictions</strong> (Unconfined).</p>
<p>(More about AppArmor in a while)</p>
<p>Connect to PinePhone over SSH and enter these commands...</p>
<pre><code class="language-bash"># Make system folders writeable and go to File Manager Click Package folder
sudo mount -o remount,rw /
cd /usr/share/click/preinstalled/.click/users/@all/com.ubuntu.filemanager

# Back up the desktop file. Restore this desktop file to go back to the normal File Manager
sudo cp com.ubuntu.filemanager.desktop com.ubuntu.filemanager.desktop.old

# Edit the desktop file
sudo nano com.ubuntu.filemanager.desktop 
</code></pre>
<p>We're now altering the behaviour of File Manager, by tampering with the <a href="http://docs.ubports.com/en/latest/appdev/platform/click.html#security-and-app-isolation"><strong>Click Package</strong></a> settings for File Manager.</p>
<p>(Why are we tampering with a Click Package? We'll learn in a while)</p>
<p>Change the <code>Exec</code> line from...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">Exec</span><span class="op">=</span><span class="ident">filemanager</span></pre></div>
<p>To...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">Exec</span><span class="op">=</span>.<span class="op">/</span><span class="ident">run</span>.<span class="ident">sh</span></pre></div>
<p>Save and exit <code>nano</code></p>
<p>We have modded the File Manager icon so that it now launches <code>run.sh</code> instead of the usual <code>filemanager</code> executable.</p>
<p>(It's like switching the executable for a Windows Explorer Shortcut)</p>
<p>We'll be installing <code>run.sh</code> later with a script: <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lvgl.sh"><code>lvgl.sh</code></a></p>
<p>In the meantime, check that <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/run.sh"><code>run.sh</code></a> (located at <code>~/lvgl-wayland/wayland</code>) contains the following...</p>
<pre><code class="language-bash"># Log Wayland messages
export WAYLAND_DEBUG=1

# Run lvgl app
./lvgl
</code></pre>
<p>If we see this...</p>
<pre><code class="language-bash"># Debug lvgl app
gdb \
    -ex=&quot;r&quot; \
    -ex=&quot;bt&quot; \
    -ex=&quot;frame&quot; \
    --args ./lvgl
</code></pre>
<p>It means that the <code>lvgl</code> app will be started with the <code>gdb</code> debugger.</p>
<p>If it crashes with a bad C pointer, the <code>gdb</code> debugger will show a helpful stack trace.</p>
<p>And this...</p>
<pre><code class="language-bash"># Run lvgl app with strace
./strace \
   -s 1024 \
   ./lvgl
</code></pre>
<p>Is for tracing the <code>lvgl</code> app with <code>strace</code>. It shows <strong>everything</strong> done by the app.</p>
<p><a href="https://github.com/lupyuen/pinephone-mir/blob/090630777782a5f4b283af84c747b9ad6c703e22/logs/filemanager-strace.log#L724">Check out this <code>strace</code> log for the File Manager on Ubuntu Touch</a></p>
<h1 id="run-lvgl-on-pinephone-with-ubuntu-touch" class="section-header"><a href="#run-lvgl-on-pinephone-with-ubuntu-touch">12 Run LVGL on PinePhone with Ubuntu Touch</a></h1>
<p>Finally we're ready to run our <code>lvgl</code> app! </p>
<p>Connect to PinePhone over SSH and enter these commands...</p>
<pre><code class="language-bash">cd ~/lvgl-wayland
./wayland/lvgl.sh
</code></pre>
<p>The script <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lvgl.sh"><code>lvgl.sh</code></a> copies <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/run.sh"><code>run.sh</code></a> from <code>~/lvgl-wayland/wayland</code> to the Click Package Folder for File Manager...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="op">/</span><span class="ident">usr</span><span class="op">/</span><span class="ident">share</span><span class="op">/</span><span class="ident">click</span><span class="op">/</span><span class="ident">preinstalled</span><span class="op">/</span>.<span class="ident">click</span><span class="op">/</span><span class="ident">users</span><span class="op">/</span>@<span class="ident">all</span><span class="op">/</span><span class="ident">com</span>.<span class="ident">ubuntu</span>.<span class="ident">filemanager</span></pre></div>
<p>In a few seconds we'll see the message...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="kw-2">*</span><span class="kw-2">*</span><span class="op">*</span> <span class="ident">Tap</span> <span class="ident">on</span> <span class="ident">File</span> <span class="ident">Manager</span> <span class="ident">icon</span> <span class="ident">on</span> <span class="ident">PinePhone</span></pre></div>
<p>Go ahead and tap the File Manager icon on PinePhone. </p>
<p>Our LVGL App shall run instead of the File Manager.</p>
<p>In the SSH console, press <code>Ctrl-C</code> to stop the log display.</p>
<p>The log file for the app is located at...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="op">/</span><span class="ident">home</span><span class="op">/</span><span class="ident">phablet</span><span class="op">/</span>.<span class="ident">cache</span><span class="op">/</span><span class="ident">upstart</span><span class="op">/</span><span class="ident">application</span><span class="op">-</span><span class="ident">click</span><span class="op">-</span><span class="ident">com</span>.<span class="ident">ubuntu</span>.<span class="ident">filemanager_filemanager_0</span>.<span class="number">7.5</span>.<span class="ident">log</span></pre></div>
<p>The log for the Wayland Compositor <code>unity-system-compositor</code> may be useful for troubleshooting...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="op">/</span><span class="ident">home</span><span class="op">/</span><span class="ident">phablet</span><span class="op">/</span>.<span class="ident">cache</span><span class="op">/</span><span class="ident">upstart</span><span class="op">/</span><span class="ident">unity8</span>.<span class="ident">log</span></pre></div>
<p>Copy the log files to our machine like this...</p>
<pre><code class="language-bash">scp -i ~/.ssh/pinephone_rsa phablet@192.168.1.160:/home/phablet/.cache/upstart/application-click-com.ubuntu.filemanager_filemanager_0.7.5.log .
scp -i ~/.ssh/pinephone_rsa phablet@192.168.1.160:/home/phablet/.cache/upstart/unity8.log .
</code></pre>
<p><a href="https://github.com/lupyuen/lvgl-wayland/blob/master/logs">Check out the sample logs</a></p>
<h1 id="overcome-apparmor-security-on-ubuntu-touch" class="section-header"><a href="#overcome-apparmor-security-on-ubuntu-touch">13 Overcome AppArmor Security on Ubuntu Touch</a></h1>
<p>To understand Wayland, AppArmor and Ubuntu Touch Security, let's look inside the script <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/wayland/lvgl.sh"><code>lvgl.sh</code></a> and discover how it launches our <code>lvgl</code> app...</p>
<ol>
<li>
<p>Our <code>lvgl</code> app doesn't have a close button, so let's terminate the app if it's already running...</p>
<pre><code class="language-bash"># Kill the app if it's already running
pkill lvgl
</code></pre>
</li>
<li>
<p>In Ubuntu Touch, <strong>User Directories</strong> (like our Home Directory) are writeable by default.</p>
<p><strong>System Directories</strong> (like <code>/usr/share</code>) are mounted with read-only access, to prevent tampering of system files. (Think malware)</p>
<p>Since we're Superuser, we may remount System Directories with read-write access...</p>
<pre><code class="language-bash"># Make system folders writeable
sudo mount -o remount,rw /
</code></pre>
</li>
<li>
<p><em>Why do we need read-write access?</em></p>
<p>Because we'll be copying our app <code>lvgl</code> and the script <code>run.sh</code> to a System Directory...</p>
<pre><code class="language-bash"># Copy app to File Manager folder
cd wayland
sudo cp lvgl /usr/share/click/preinstalled/.click/users/@all/com.ubuntu.filemanager

# Copy run script to File Manager folder
sudo cp run.sh /usr/share/click/preinstalled/.click/users/@all/com.ubuntu.filemanager
</code></pre>
</li>
<li>
<p><em>What's this folder <code>/usr/share/click/preinstalled/.click/users/@all/com.ubuntu.filemanager</code>?</em></p>
<p>This is the <a href="http://docs.ubports.com/en/latest/appdev/platform/click.html#security-and-app-isolation"><strong>Click Package</strong></a> folder for File Manager.</p>
<p>Ubuntu Touch Apps (like File Manager) are packaged as Click Packages for installation on our phones.</p>
<p>When the app is installed, Ubuntu Touch extracts the Click Package into a folder under <code>/usr/share/click</code>.</p>
<p>Inside the Click Package folder we'll find the executables, libraries and data files that are needed for running the app.</p>
<p>The folder also contains a <code>.desktop</code> file. (Earlier we've seen <code>com.ubuntu.filemanager.desktop</code> for File Manager) This file tells Ubuntu Touch how to launch the app.</p>
</li>
<li>
<p><em>Does the app run as our user account <code>phablet</code>?</em></p>
<p>Nope. For security, Ubuntu Touch Apps run under an account with restricted privileges: <code>clickpkg</code></p>
<p>This account has no access to our <code>phablet</code> files. That's why we copy the <code>lvgl</code> app and <code>run.sh</code> script to the Click Package folder, which is accessible by <code>clickpkg</code></p>
<p>We set the ownership of <code>lvgl</code> and <code>run.sh</code> to <code>clickpkg</code> so that it can execute the files...</p>
<pre><code class="language-bash"># Set ownership on the app and the run script
sudo chown clickpkg:clickpkg /usr/share/click/preinstalled/.click/users/@all/com.ubuntu.filemanager/lvgl
sudo chown clickpkg:clickpkg /usr/share/click/preinstalled/.click/users/@all/com.ubuntu.filemanager/run.sh
</code></pre>
</li>
<li>
<p>Our <code>lvgl</code> app and <code>run.sh</code> script have been staged in the Click Package folder.</p>
<p>We ask the human to tap the File Manager icon...</p>
<pre><code class="language-bash"># Start the File Manager
echo &quot;*** Tap on File Manager icon on PinePhone&quot;
</code></pre>
</li>
<li>
<p>Ubuntu Touch launches our <code>lvgl</code> app. As our app runs, it logs debugging messages to Standard Output and Standard Error.</p>
<p>The messages are captured in this log file...</p>
<pre><code class="language-bash"># Monitor the log file
echo &gt;/home/phablet/.cache/upstart/application-click-com.ubuntu.filemanager_filemanager_0.7.5.log
tail -f /home/phablet/.cache/upstart/application-click-com.ubuntu.filemanager_filemanager_0.7.5.log
</code></pre>
</li>
</ol>
<p><em>Why can't we run our <code>lvgl</code> app from the Terminal Command Line?</em></p>
<p>Because Ubuntu Touch's Wayland Service stops unauthorized processes from grabbing the Compositor...</p>
<p><img src="https://lupyuen.github.io/images/wayland-security.jpg" alt="Stopped by Wayland Security" /></p>
<p>We see this in the Wayland Compositor log: <code>/home/phablet/.cache/upstart/unity8.log</code></p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">ApplicationManager</span> <span class="ident">REJECTED</span> <span class="ident">connection</span> <span class="ident">from</span> <span class="ident">app</span> <span class="ident">with</span> <span class="ident">pid</span> <span class="number">6710</span> 
<span class="kw">as</span> <span class="ident">it</span> <span class="ident">was</span> <span class="ident">not</span> <span class="ident">launched</span> <span class="ident">by</span> <span class="ident">upstart</span>, <span class="ident">and</span> <span class="ident">no</span> <span class="ident">desktop_file_hint</span> <span class="ident">is</span> <span class="ident">specified</span></pre></div>
<p>That's why we need to inject <code>lvgl</code> into File Manager... So that Wayland thinks that the File Manager is grabbing the Compositor.</p>
<p><em>Why did we choose the File Manager app instead of another app like Camera?</em></p>
<p>Because File Manager has Unconfined AppArmor Permissions... It can do anything! </p>
<p>(But still restricted by the <code>clickpkg</code> user permissions)</p>
<p>Look at the <a href="http://docs.ubports.com/en/latest/appdev/platform/apparmor.html">AppArmor Policy</a> for the File Manager App: <a href="https://gitlab.com/ubports/apps/filemanager-app/-/blob/master/filemanager.apparmor"><code>filemanager.apparmor</code></a></p>
<pre><code class="language-json">{
    &quot;policy_version&quot;: 16.04,
    &quot;template&quot;: &quot;unconfined&quot;,
    &quot;policy_groups&quot;: []
}
</code></pre>
<p>Compare this with the AppArmor Policy for the Camera App: <a href="https://gitlab.com/ubports/apps/camera-app/-/blob/master/camera.apparmor"><code>camera.apparmor</code></a></p>
<pre><code class="language-json">{
    &quot;policy_groups&quot;: [
        &quot;picture_files&quot;,
        &quot;video_files&quot;,
        &quot;camera&quot;,
        &quot;audio&quot;,
        &quot;video&quot;,
        &quot;usermetrics&quot;,
        &quot;content_exchange&quot;,
        &quot;content_exchange_source&quot;,
        &quot;location&quot;
    ],
    &quot;policy_version&quot;: 16.04,
    &quot;read_path&quot;: [
        &quot;@{PROC}/*/mounts&quot;,
        &quot;/dev/disk/by-label/&quot;
    ]
}
</code></pre>
<p>The AppArmor Policy says that the Camera App may only access selected features (like recording audio and video). And it's only allowed to read specific paths (like <code>/dev/disk/by-label</code>).</p>
<p><code>strace</code> won't work with the AppArmor Policy for Camera App. </p>
<p>So for tracing our app with <code>strace</code>, we &quot;borrow&quot; the Unconfined AppArmor Policy for File Manager.</p>
<p>To troubleshoot problems with AppArmor, check the system log in <code>/var/log/syslog</code></p>
<p><a href="https://github.com/lupyuen/lvgl-wayland/blob/master/logs/syslog">Check out my <code>syslog</code></a></p>
<h1 id="what-i-like-about-ubuntu-touch-on-pinephone" class="section-header"><a href="#what-i-like-about-ubuntu-touch-on-pinephone">14 What I like about Ubuntu Touch on PinePhone</a></h1>
<p>While attempting to port the <a href="https://lupyuen.github.io/pinetime-rust-mynewt/articles/gotk3">PineTime Companion App</a> to PinePhone with GTK (<a href="https://github.com/lupyuen/pinephone-mir#wayland-messages-for-gtk-app">and failing miserably</a>), I had these thoughts...</p>
<ol>
<li>
<p><strong>AppArmor is good</strong>, because iOS and Android have similar apps security</p>
</li>
<li>
<p><strong>Read-only file system is good</strong> (system files are read-only by default, user files are read-write). Helps to prevent security holes. (Even PineTime has a read-only Flash ROM)</p>
</li>
<li>
<p><strong>Why is Qt supported on Ubuntu Touch and not GTK?</strong> Because building a Linux mobile app requires mobile-friendly widgets. </p>
<p>I think Qt has more mobile-friendly widgets, even through the internal plumbing is way too complicated. </p>
<p>When I get GTK running on Ubuntu Touch, I will face the same problem with widgets. And I have to make GTK widgets look and feel consistent with Qt / Ubuntu Touch widgets.</p>
<p>That's why I decided to move away from GTK and focus on a simpler widget framework with LVGL.</p>
</li>
<li>
<p><strong>Older kernel base</strong> in Ubuntu Touch... I don't do kernel hacking much so it doesn't matter to me. </p>
<p>I think for mobiles we only need to support a few common chipsets, so an older kernel is probably fine. </p>
<p>That explains why Raspberry Pi 4 isn't supported by Ubuntu Touch... The hardware is just too new.</p>
</li>
<li>
<p>The issues I'm struggling with now... Wayland, GTK3, ... are actually really old stuff. Updating the kernel won't help.</p>
</li>
<li>
<p><strong>Ubuntu Touch is pure Wayland</strong>, none of the legacy X11 stuff. Xwayland is not even there (unless you use the Libertine containers ugh). </p>
<p>The pure Wayland environment causes GTK to break, because GTK assumes some minimal X11 support (i.e. Xwayland).</p>
<p>It's better to start from scratch with a toolkit that's not based on X11, like LVGL.</p>
</li>
<li>
<p>So Ubuntu Touch is not really that bad for PinePhone... It's just painful for building non-Qt apps. 🙂</p>
</li>
</ol>
<p>After <a href="https://twitter.com/UBports/status/1282934927806398464">posting my thoughts</a>, the UBports, GNOME and Xfce Community responded with encouraging and insightful notes...</p>
<h2 id="ubports-on-ubuntu-touch-wayland-and-mir" class="section-header"><a href="#ubports-on-ubuntu-touch-wayland-and-mir">14.1 UBports on Ubuntu Touch, Wayland and Mir</a></h2>
<ul>
<li>
<p><a href="https://twitter.com/UBports/status/1282936886311428096?s=09">Ubuntu Touch supports Wayland only on non-Android devices</a> like PinePhone</p>
</li>
<li>
<p><a href="https://twitter.com/UBports/status/1282934927806398464?s=09">Ubuntu Touch uses older kernels on Android devices</a>, only because that's the only way to run on those devices. PinePhone uses a newer kernel.</p>
</li>
<li>
<p><a href="https://twitter.com/UBports/status/1282936886311428096">UBports is fixing that with Halium9</a>, a Hardware Abstraction Layer based on parts of AOSP 9 and other hardware enablement components</p>
</li>
</ul>
<p><a href="https://www.phoronix.com/scan.php?page=news_item&amp;px=Mir-2019-Kicking">Read about Unity8 / Mir / Lomiri's complicated history</a></p>
<p><a href="https://www.phoronix.com/scan.php?page=news_item&amp;px=Unity-8-Renamed-To-Lomiri">Another article</a></p>
<h2 id="gnome-and-gtk-on-wayland" class="section-header"><a href="#gnome-and-gtk-on-wayland">14.2 GNOME and GTK on Wayland</a></h2>
<ul>
<li>
<p><a href="https://mastodon.social/@ebassi/104511735257435944">GNOME Shell developers are working into making X11 completely optional</a></p>
</li>
<li>
<p>GTK can already be built without X11 backend</p>
</li>
</ul>
<h2 id="wayland-on-xfce" class="section-header"><a href="#wayland-on-xfce">14.3 Wayland on Xfce</a></h2>
<ul>
<li>
<p><a href="https://twitter.com/XfceNation/status/1284842929895301120?s=09">Proprietary Nvidia graphics drivers didn't have Wayland support out of the box</a> as they wanted to use EGLStreams rather than GBM. So Nvidia did the work to get GNOME and KDE to support EGLStreams.</p>
</li>
<li>
<p><a href="https://twitter.com/XfceNation/status/1284854254092513285?s=09">Only GNOME and KDE have Wayland support and the support isn't feature complete compared to X11</a>, which is why most distros with GNOME and KDE editions still do not default to Wayland</p>
</li>
</ul>
<h1 id="touch-input-for-lvgl-on-wayland" class="section-header"><a href="#touch-input-for-lvgl-on-wayland">15 Touch Input for LVGL on Wayland</a></h1>
<p>I'm sorry the port of LVGL on Wayland to PinePhone is incomplete, and I am now working on other PineTime Smart Watch projects.</p>
<p>The Touch Input needs to be ported for LVGL like this...</p>
<ul>
<li><a href="https://docs.lvgl.io/latest/en/html/porting/indev.html">LVGL Input Device Interface</a></li>
</ul>
<p>The LVGL Input Driver needs to be implemented with Wayland (without X) like this...</p>
<ul>
<li>
<p><a href="https://jan.newmarch.name/Wayland/Input/">Programming Wayland Clients - Input</a></p>
</li>
<li>
<p><a href="https://wayland-book.com/seat.html">The Wayland Protocol - Seats: Handling input</a></p>
</li>
</ul>
<p>Also please take care of the LVGL Tick Interface. We need to call <code>lv_tick_inc()</code> periodically or our LVGL Display Driver won't flush when calling <code>lv_task_handler()</code>...</p>
<ul>
<li><a href="https://docs.lvgl.io/latest/en/html/porting/tick.html">LVGL Tick Interface</a></li>
</ul>
<p>I noticed this behaviour so I called <code>lv_tick_inc(100)</code> before <code>lv_task_handler()</code>...</p>
<ul>
<li><a href="https://github.com/AppKaki/lvgl-wasm/blob/master/wasm/lvgl.c#L81-L88"><code>github.com/AppKaki/lvgl-wasm/wasm/lvgl.c</code></a></li>
</ul>
<p>That's the code I used for my latest project, a WebAssembly LVGL Simulator for PineTime Smart Watch...</p>
<ul>
<li>
<p><a href="https://github.com/AppKaki/lvgl-wasm/blob/master/README.md"><code>lvgl-wasm</code>: PineTime Watch Face Simulator with LVGL ported to WebAssembly</a></p>
</li>
<li>
<p><a href="https://lupyuen.github.io/pinetime-rust-mynewt/articles/simulator">Preview PineTime Watch Faces in your Web Browser with WebAssembly</a></p>
</li>
</ul>
<h1 id="whats-next" class="section-header"><a href="#whats-next">16 What's Next?</a></h1>
<p>Wayland feels like New Underwear... And it needs a New App Toolkit like LVGL to make us comfortable.</p>
<p>If you're keen to help make <strong>LVGL the Newer, Simpler App Toolkit</strong> on Wayland, Ubuntu Touch and PinePhone, <a href="https://github.com/lupyuen">please lemme know</a>! 😀</p>
<p><a href="https://lupyuen.github.io/rss.xml">Check out my RSS Feed</a></p>
<h1 id="configure-ssh-on-pinephone" class="section-header"><a href="#configure-ssh-on-pinephone">17 Configure SSH on PinePhone</a></h1>
<p><strong>First Thing</strong> to do when we get our new PinePhone: Open the PinePhone Back Cover and <a href="https://twitter.com/RealDanct12/status/1286969529516453888?s=20"><strong>Remove the Battery Insulation Sticker!</strong></a></p>
<p>(Can't believe I missed that!)</p>
<p><strong>Second Thing</strong>: Protect the SSH Service on PinePhone with <strong>SSH Keys</strong>. And start the SSH Service only when necessary.</p>
<p>Here's how...</p>
<h2 id="generate-ssh-keys" class="section-header"><a href="#generate-ssh-keys">17.1 Generate SSH Keys</a></h2>
<ol>
<li>
<p>On our Computer (not PinePhone), open a Command Prompt. Enter this (and fill in our email)...</p>
<pre><code class="language-bash">ssh-keygen -t rsa -b 4096 -C &quot;your_email@example.com&quot;
</code></pre>
</li>
<li>
<p>When prompted...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">Enter</span> <span class="ident">a</span> <span class="ident">file</span> <span class="kw">in</span> <span class="ident">which</span> <span class="ident">to</span> <span class="ident">save</span> <span class="ident">the</span> <span class="ident">key</span></pre></div>
<p>Press <code>Enter</code>. This stores the new SSH Key in the <code>.ssh</code> folder in our Home Directory.</p>
</li>
<li>
<p>When prompted...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">Enter</span> <span class="ident">a</span> <span class="ident">file</span> <span class="kw">in</span> <span class="ident">which</span> <span class="ident">to</span> <span class="ident">save</span> <span class="ident">the</span> <span class="ident">key</span></pre></div>
<p>Enter...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">pinephone_rsa</span></pre></div>
<p>We'll create an SSH Key Pair named <code>pinephone_rsa</code> (Private Key) and <code>pinephone_rsa.pub</code> (Public Key)</p>
</li>
<li>
<p>When prompted...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">Enter</span> <span class="ident">passphrase</span></pre></div>
<p>Press <code>Enter</code>. We won't need a passphrase unless our PinePhone needs to be super-secure.</p>
</li>
</ol>
<p>This creates an SSH Key Pair in the <code>.ssh</code> folder in our Home Directory...</p>
<ul>
<li>
<p><code>pinephone_rsa</code> contains the Private Key. Never give the Private Key to others!</p>
</li>
<li>
<p><code>pinephone_rsa.pub</code> contains the Public Key. We'll copy this to PinePhone now.</p>
</li>
</ul>
<p>(<a href="https://docs.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generating-a-new-ssh-key">Adapted from this doc</a>)</p>
<h2 id="install-ssh-keys" class="section-header"><a href="#install-ssh-keys">17.2 Install SSH Keys</a></h2>
<ol>
<li>
<p>Copy <code>pinephone_rsa.pub</code> from the <code>.ssh</code> folder in our Home Directory to a MicroSD Card.</p>
</li>
<li>
<p>Insert the MicroSD Card into PinePhone. Copy <code>pinephone_rsa.pub</code> to our Home Directory on PinePhone.</p>
<p>(Check the section &quot;Copy Files from MicroSD Card on PinePhone&quot; below)</p>
</li>
<li>
<p>Tap the Terminal icon on PinePhone. Enter...</p>
<pre><code class="language-bash"># Go to home directory
cd

# If .ssh folder doesn't exist, create it
mkdir .ssh
chmod 700 .ssh

# Set public key as the authorized key
cp pinephone_rsa.pub .ssh/authorized_keys
chmod 600 .ssh/authorized_keys

# Show the SSH files
ls -la ~/.ssh
</code></pre>
<p>We should see this...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">drwx</span><span class="op">-</span><span class="op">-</span><span class="op">-</span><span class="op">-</span><span class="op">-</span><span class="op">-</span>  <span class="number">2</span> <span class="ident">phablet</span> <span class="ident">phablet</span> <span class="number">4096</span> <span class="ident">Jul</span>  <span class="number">7</span> <span class="number">20</span>:<span class="number">06</span> .
<span class="ident">drwxr</span><span class="op">-</span><span class="ident">xr</span><span class="op">-</span><span class="ident">x</span> <span class="number">28</span> <span class="ident">phablet</span> <span class="ident">phablet</span> <span class="number">4096</span> <span class="ident">Jul</span> <span class="number">24</span> <span class="number">11</span>:<span class="number">38</span> ..
<span class="op">-</span><span class="ident">rw</span><span class="op">-</span><span class="op">-</span><span class="op">-</span><span class="op">-</span><span class="op">-</span><span class="op">-</span><span class="op">-</span>  <span class="number">1</span> <span class="ident">phablet</span> <span class="ident">phablet</span>  <span class="number">743</span> <span class="ident">Jul</span>  <span class="number">7</span> <span class="number">20</span>:<span class="number">08</span> <span class="ident">authorized_keys</span></pre></div>
<p>Check that the permissions (<code>rw</code>) and owner (<code>phablet</code>) are correct.</p>
</li>
</ol>
<h2 id="start-ssh-service" class="section-header"><a href="#start-ssh-service">17.3 Start SSH Service</a></h2>
<p>To start the SSH Service on PinePhone, open the Terminal app.</p>
<p>Create a file named <code>a</code>...</p>
<pre><code class="language-bash">nano a
</code></pre>
<p>Type this script into the <code>a</code> file...</p>
<pre><code class="language-bash">#!/bin/sh
# Script to start SSH service and show IP address

# Start SSH service
sudo service ssh start

# Show IP address
ifconfig | \
    grep -v &quot;127.0.0.1&quot; | \
    grep &quot;inet addr:&quot;

# Ping repeatedly to keep WiFi alive
ping google.com
</code></pre>
<p>Save the file and exit <code>nano</code>.</p>
<p>(Or download the file from <a href="https://github.com/lupyuen/lvgl-wayland/blob/master/a"><code>lvgl-wayland/a</code></a> and copy via a MicroSD Card. Check the next section for instructions.)</p>
<p>When we're ready do coding on PinePhone, enter this at the Terminal command line...</p>
<pre><code class="language-bash">. a 
</code></pre>
<p>(There's a space between &quot;<code>.</code>&quot; and &quot;<code>a</code>&quot;)</p>
<p>The script starts the SSH Service and displays the IP address for PinePhone...</p>
<p><img src="https://lupyuen.github.io/images/wayland-ssh.jpg" alt="Starting SSH Service on PinePhone" /></p>
<p>From our Computer, we'll connect to PinePhone at the IP adddress indicated by <code>inet addr</code>, say <code>192.168.1.160</code>...</p>
<pre><code class="language-bash">ssh -i ~/.ssh/pinephone_rsa phablet@192.168.1.160
</code></pre>
<p>And that's how we access PinePhone via SSH!</p>
<p>When we press PinePhone's power button to switch off PinePhone, we'll see ths amusing message from olden times...</p>
<p><img src="https://lupyuen.github.io/images/wayland-halt.jpg" alt="Powering off PinePhone" /></p>
<p>If typing on a touch keyboard is not your thing, try copying the files from a MicroSD card...</p>
<p><img src="https://lupyuen.github.io/images/wayland-sd.jpg" alt="How we insert a MicroSD Card into PinePhone at night" /></p>
<p><em>How we insert a MicroSD Card into PinePhone at night</em></p>
<h1 id="copy-files-from-microsd-card-on-pinephone" class="section-header"><a href="#copy-files-from-microsd-card-on-pinephone">18 Copy Files from MicroSD Card on PinePhone</a></h1>
<p>It's useful to transfer files to PinePhone via MicroSD Card, like SSH Keys and the SSH Script <code>a</code> above.</p>
<p>(Sadly PinePhone on Ubuntu Touch doesn't allow receiving files over Bluetooth)</p>
<p>The MicroSD card on PinePhone doesn't appear in the File Manager unless we mount it.</p>
<p>Tap the Terminal icon on PinePhone and enter the following...</p>
<pre><code class="language-bash">ls -l /dev/disk/by-label
</code></pre>
<p>We should see something like this...</p>

<div class="example-wrap"><pre class="rust rust-example-rendered">
<span class="ident">lrwxrwxrwx</span> <span class="number">1</span> <span class="ident">root</span> <span class="ident">root</span> <span class="number">15</span> <span class="ident">Jul</span> <span class="number">23</span> <span class="number">22</span>:<span class="number">24</span> <span class="ident">BOOT_MNJRO</span> <span class="op">-&gt;</span> ..<span class="op">/</span>..<span class="op">/</span><span class="ident">mmcblk0p1</span>
<span class="ident">lrwxrwxrwx</span> <span class="number">1</span> <span class="ident">root</span> <span class="ident">root</span> <span class="number">15</span> <span class="ident">Jul</span> <span class="number">23</span> <span class="number">22</span>:<span class="number">24</span> <span class="ident">cache</span> <span class="op">-&gt;</span> ..<span class="op">/</span>..<span class="op">/</span><span class="ident">mmcblk2p8</span>
<span class="ident">lrwxrwxrwx</span> <span class="number">1</span> <span class="ident">root</span> <span class="ident">root</span> <span class="number">15</span> <span class="ident">Jul</span> <span class="number">23</span> <span class="number">22</span>:<span class="number">24</span> <span class="ident">ROOT_MNJRO</span> <span class="op">-&gt;</span> ..<span class="op">/</span>..<span class="op">/</span><span class="ident">mmcblk0p2</span>
<span class="ident">lrwxrwxrwx</span> <span class="number">1</span> <span class="ident">root</span> <span class="ident">root</span> <span class="number">16</span> <span class="ident">Jul</span> <span class="number">23</span> <span class="number">22</span>:<span class="number">24</span> <span class="ident">userdata</span> <span class="op">-&gt;</span> ..<span class="op">/</span>..<span class="op">/</span><span class="ident">mmcblk2p10</span></pre></div>
<p>These are the Partition Labels on our MicroSD Card. </p>
<p>Let's say we wish to mount the MicroSD Card partition <code>ROOT_MNJRO</code>, which links to <code>/dev/mmcblk0p2</code>...</p>
<pre><code class="language-bash">mkdir /tmp/sdcard
sudo mount /dev/mmcblk0p2 /tmp/sdcard
ls -l /tmp/sdcard
</code></pre>
<p>(If we don't see our Patition Label, try mounting the numbered partitions anyway: <code>/dev/mmcblk0p1</code>, <code>p2</code>, <code>p3</code>, ...)</p>
<p>We should see the contents of our MicroSD Card.</p>
<p>The MicroSD Card will now appear in File Manager as <code>/tmp/sdcard</code>, ready for us to copy the files.</p>
<p>Or just copy files from the Command Line like so...</p>
<pre><code class="language-bash">cp /tmp/sdcard/a ~
</code></pre>
<p>When we're done, unmount our MicroSD Card...</p>
<pre><code class="language-bash">sudo umount /tmp/sdcard
</code></pre>
<p><img src="https://lupyuen.github.io/images/wayland-weston.png" alt="LVGL App on Pinebook Pro" /></p>
<h1 id="build-and-test-lvgl-app-on-linux" class="section-header"><a href="#build-and-test-lvgl-app-on-linux">19 Build and Test LVGL App on Linux</a></h1>
<p>Our LVGL App works on Linux machines like Pinebook Pro...</p>
<pre><code class="language-bash"># Download the source code
git clone https://github.com/lupyuen/lvgl-wayland
cd lvgl-wayland

# Build the lvgl executable
make

# Install Weston Wayland Compositor...
# For Arch Linux and Manjaro:
sudo pacman -S weston

# For Other Distros:
# Check https://github.com/wayland-project/weston

# Start the Weston Wayland Compositor with the PinePhone screen dimensions
weston --width=720 --height=1398 &amp;

# Run the lvgl executable
./wayland/lvgl
</code></pre>

    
</body>
</html>